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Abstract: Physicians in the 21st century will increasingly interact in diverse ways with 
information systems, requiring competence in many aspects of clinical informatics. In recent 
years, many medical school curricula have added content in information retrieval (search) and 
basic use of the electronic health record. However, this omits the growing number of other 
ways that physicians are interacting with information that includes activities such as clinical 
decision support, quality measurement and improvement, personal health records, telemedicine, 
and personalized medicine. We describe a process whereby six faculty members representing 
different perspectives came together to define competencies in clinical informatics for a curricu- 
lum transformation process occurring at Oregon Health & Science University. From the broad 
competencies, we also developed specific learning objectives and milestones, an implementa- 
tion schedule, and mapping to general competency domains. We present our work to encourage 
debate and refinement as well as facilitate evaluation in this area. 

Keywords: curriculum transformation, clinical decision support, patient safety, health care 
quality, patient engagement 

Introduction 

Physicians and medical students have been using health information technology (HIT) 
for decades. During this time, the role of HIT has changed dramatically from a useful 
tool for data access and occasional information retrieval to a ubiquitous presence 
that permeates health care and medical practice in myriad ways. As the use of HIT 
has accelerated and the underlying science of biomedical and health informatics has 
advanced, medical education has lagged behind, leaving physicians and students alike 
to learn largely on their own how to make use of these tools. While some medical 
schools have introduced limited aspects of biomedical informatics into their curricula, 
most of this is focused on training to perform basic tasks such as accessing knowledge 
resources or basic use of an electronic health record (EHR). 

Twenty-first century clinicians face a much more digital world than their predecessors. 
The quantity of biomedical knowledge continues to expand, with an attendant increase 
in the primary scientific literature. 1 Secondary sources that summarize this information 
proliferate as well, not only for use by clinicians but also by patients and consumers. 
Programs such as the "meaningful use" financial incentives of the Health Information 
for Technology and Clinical Health Act require comprehensive adoption and use of the 
EHR. 2,3 Patients, especially the aging "baby boomer" generation, want to interact with 
the health care system the same way they interact with airlines, banks, and retailers, ie, 
through digital means using technologies such as the personal health record. 4 Patients, 
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payers, and purchasers demand more accountability in health 
care quality, safety, and cost, 5 leading to an expectation of 
measurement and reporting of quality of care as a routine part 
of participation in new delivery mechanisms such as primary 
care medical homes and accountable care organizations. At 
the same time, telemedicine applications extend the reach of 
health care systems and clinicians in both rural and urban 
settings. Ideally, for all of these applications, patient data 
must move readily across organizational boundaries via health 
information exchange, while privacy and confidentiality are 
protected. 6 The growing quantity of clinical and administrative 
data in these systems also affords an opportunity for advanced 
analysis that can enable better deployment of resources and 
coordination of care, facilitation of personalized and preci- 
sion medicine, and advancement of clinical and translational 
research. 7 Together, these advances are moving health care 
toward the global vision of the learning health system put 
forth by the Institute of Medicine (IOM). 8 - 9 

The importance of these developments is further 
evidenced by the recent establishment of a new medical 
subspecialty of clinical informatics. 10 Practicing physicians 
are now beginning to become board-certified in this new 
subspecialty, with fellowship programs accredited by the 
Accreditation Council for Graduate Medical Education 
(ACGME) to soon be established. This underscores the need 
for medical students to be introduced to the concepts and 
competencies of this new medical subspecialty as part of the 
undergraduate curriculum. 

In this paper, we describe our work to expand the under- 
graduate medical curriculum to include a comprehensive set 
of competencies in clinical informatics. We go beyond the 
usual focus on information retrieval from knowledge sources 
and data access through the EHR, aiming to address the 
expanded and diverse roles of HIT in contemporary health 
care delivery, personal health, public health, and clinical 
and translational research. We describe the development 
of a novel biomedical informatics curriculum that is being 
implemented within a comprehensive transformation of 
the undergraduate curriculum at our institution. Our future 
plans include evaluation of the impact of this work, and we 
encourage others to participate in this evaluation and refine- 
ment process. 

Previous work developing 
competencies in informatics 

The need to include biomedical informatics in medical 
education has been recognized in previous publications of 
competencies, curricula, and other learning materials for 



informatics. One early, high-profile resource was part of the 
Medical Student Objectives Project of the Association of 
American Medical Colleges, which framed proposed objec- 
tives in terms of five key roles of physicians: clinicians, 
life-long learners, educators, researchers, and managers. 11 
Although current medical students believe these objectives 
are important to their education, 12 uptake of these objectives 
has been modest. 13 More recently, the Alliance for Clinical 
Education issued a policy statement calling for competen- 
cies related to the EHR, 14 although also noting that many 
challenges to implementing such competencies exist. 15 
Others have also noted the challenges to education about 
the EHR, citing problems analogous to those of clinicians 
incorporating EHRs into practice, such as integration into 
workflows, and concerns about unanticipated and undesir- 
able impacts. 16 It is clear, however, that competencies in the 
use of the EHR are aligned with more general physician 
educational competencies, 1718 and that informatics curricula 
can be incorporated into undergraduate medical training 19 20 
and evaluated. 21 

Methods 

In 2012, the Oregon Health & Science University, School 
of Medicine began a comprehensive transformation of the 
undergraduate medical curriculum for the 21st century. 
From the outset, the Dean and school leadership recognized 
the importance of informatics to physicians and health 
care in the future. With participation and support from the 
Senior Associate Dean for Education (GM), a team of five 
physician-faculty was convened to lead this effort, including 
the following: 

• An informatician involved nationally in informatics 
education leadership and development of the new clinical 
informatics subspecialty (WH) 

• An internist-informatician who is chair of the medical 
school curriculum committee and actively engaged in 
the new curriculum development team (PG) 

• A family physician and innovator in using simula- 
tion for teaching and assessing EHR use by medical 
students (FB) 

• A critical care physician investigator using EHR simula- 
tion to assess residents' ability to detect important patterns 
in critical care data (JG) 

• An informatician and former internal medicine residency 
associate program director also involved in the previous 
two above activities (VM) 

Using an exploratory qualitative methodology based on 
action research and working with the curriculum transformation 
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leadership, we developed a broad set of competencies in 
informatics for medical education. With each competency, 
we developed a set of learning objectives, a designation of 
when it would be most appropriate to introduce the concept 
during the medical school curriculum, and how each mapped 
to the broader ACGME competencies. 

Results 

Developing competencies 
and learning objectives 

Before beginning to identify the competencies for student 
learning, we needed to develop the rationale for including 
these topics in the curriculum. This led us to map the com- 
petencies to the six ACGME core competency domains 22 and 
develop explicit learning objectives within each competency, 
followed by their implementation in the emerging curriculum. 
Discerning the rationale for our competencies was a narrative 
process, with their implementation enumerating the specific 
areas deemed important to incorporate into the medical 
school curriculum, as it was being concurrently designed. 

Our first step was to provide the rationale to include vari- 
ous topic areas in the curriculum. A first critical concept we 
agreed upon was that informatics is not the same as computer 
literacy. Computer literacy is one of many requirements 
to use informatics successfully, but knowing how to use a 
computing device (personal computer, tablet, or smartphone) 
is not the same as having skills in informatics, ie, using that 
device to improve health, health care delivery, public health, 
or research. 

We concurred that one fundamental skill currently being 
taught in many medical schools for 21st century clinicians 
was something taught in many medical schools since the 
late 20th century, namely, how to find information to apply 
to patient care. The needed informatics skills start, and do 
not stop, with the knowledge of how to enter simple queries 
into common general (eg, Google™, http://www. google, 
com ) or medical (eg, PubMed, http : //www.pubmed .gov ) 
search engines. Clinicians must also know how to formu- 
late a clinical question as an answerable one, and then be 
able to select the appropriate resource and make optimal 
use of it. This includes knowing what content is in different 
search systems. He or she must also know about specialized 
resources such as the Agency for Healthcare Research and 
Quality Guidelines Clearinghouse ( http ; //www, guideline . 
gov ) and the Centers for Disease Control and Prevention 
travel site ( http://wwwnc.cdc.gov/travel ). Clinicians should 
also have an understanding of the major commercial pub- 
lishers as well as what professional societies offer. Once 



clinicians know how to use a search site, they must be able 
to phrase an appropriate query and know how to maximize 
the search using advanced query features. To take maximum 
advantage of available knowledge resources, it is necessary 
to understand the advanced search features that are available, 
whether in general web resources such as Google or dedicated 
biomedical knowledge resources such as PubMed. 

Once information is retrieved, clinicians must know how 
to critically appraise the information retrieved and decide, 
if warranted, to apply it to the patient or population. As with 
searching, the type of appraisal varies with the search engine 
used. With output from general search engines like Google, 
the trustworthiness of the information must be assessed. 
Clinicians must also know how to make optimal use of 
patient data and information. They must know how to use 
informatics to achieve the triple aim of better health, better 
care, and lower cost. 5 An appropriate framework for applying 
this emanates from the IOM concept of the "learning health 
system", 8 - 9 which presents a compelling vision for a health 
care system that is patient-centered, population-based, and 
promotes learning from data. 

Being able to use data and information also means under- 
standing that the patient health record is more than "charting", 
and that its value goes beyond being able to read it. Clinicians 
must be facile with all aspects of the EHR, able to easily move 
from one vendor system to another and to understand the critical 
role of health information exchange in making any single record 
as complete as possible. 6 This issue is magnified by the increas- 
ing amount of data the clinicians must view and manage. 23 
Further, as the health care focus moves from one of quantity 
to one of value, clinicians must pay attention to quality, patient 
safety, and cost. This requires coordination of care, in not just 
providing medical procedures, nursing interventions, therapies, 
etc in isolation, but also teamwork and communication. 24 

From this vantage, the health record is no longer a 
passive collection of information used mainly to justify 
billing. Rather, it is a source of data, organized into coherent 
information that allows the health care team to deliver the 
best, safest, and most cost-effective care. This means that 
the 21st century clinician must have a basic understanding 
of informatics issues, such as capturing data that is correct 
and complete as well as consistent in its expression. 25 He 
or she must be able to work in partnership with informatics 
professionals to achieve what we know is so critical in the 
application of informatics: adhering to standards, achieving 
system interoperability, appropriately and optimally imple- 
menting clinical decision support, and maintaining security 
to assure privacy and confidentiality. 26 
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Clinicians also require understanding of areas of quality 
measurement and improvement, especially as it applies to 
their direct practice. 5 The clinician must understand concepts 
of health care quality and how the EHR and other sources 
of data are increasingly used to measure it. They must also 
know how to work with clinical leaders to select, implement, 
and improve quality measures. This should be introduced as 
part of the larger learning health system. 8 9 This discussion 
should also include the related issues of patient safety and 
medical errors, 27 including safety of HIT. 28 

Patient engagement is another component for safe, 
effective, and coordinated care. 29 Not only does patient 
engagement improve health care, but patients, including 
Internet-savvy baby boomers, will increasingly demand 
it. Patients want health care that adapts the online features 
of other modern industries, allowing them the ability to 
view their own data and interact with their clinicians and 
health care system (eg, online scheduling of appointments, 
prescription refills, and even consultations that are appropri- 
ate via the Internet or other electronic media). 29 These will 
likely take the form of a personal health record, accessible 
from a patient portal that allows access to all information, 
not just that from the system of the provider organization. 4 
Because of this natural evolution of health care, clinicians 
must also have knowledge and understanding of the appro- 
priate use of telemedicine and telehealth (such as e-visits), 
both for remote locations and as a convenient option locally 
in patients' homes and other settings. 

Complementary to the patient-centric view, clinicians 
must also understand population-based care 30 and the infor- 
matics underlying it. 31 The clinician and their team care for 
populations of patients; they must be able to view their care 
needs and results across their patient population. When a 
new test or treatment is determined to be highly effective, 
the clinician must be able to quickly identify patients who are 
candidates for it. They must also be able to identify outliers 
in their populations who require intervention, such as those 
with excessively high blood pressure or blood sugar, missed 
appointments or screen tests, or those at risk for hospital 
(re)admission. 

Another area where 2 1 st century clinicians must under- 
stand the key issues is in bioinformatics, especially as it 
relates to personalized medicine. 32 Modern clinicians need not 
understand complicated gene sequencing algorithms, but they 
should have an understanding of how genomics and related 
areas are transforming our understanding of maintaining health 
and diagnosing and treating disease. 33 Now that the vision of 
personalized medicine is coming to fruition, clinicians will 
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increasingly rely on the help of decision support and other 
tools for assistance in applying it to individual patients. He 
or she should have a basic understanding of genome-wide 
association studies and their ramifications. 34 

Clinicians must also understand the strengths and limita- 
tions of clinical and translational research. They must under- 
stand the differences between and the value of experimental 
and observational studies. Ideally, students will have partici- 
pated in research while in their training. But even if not, they 
should understand issues such as data quality, study design, 
and the limitations that come from the sharp focus perspective 
of a clinical study. Students should participate in the learning 
health system laid out in the vision of groups like the IOM. 

Implementing competencies 
and objectives 

Based on the above narrative, our group of diverse clinician- 
educators developed through an iterative process a set of 
13 competencies in clinical informatics (Table 1). Each 
competency was mapped to one or more of the six ACGME 
general competency domains. We then developed more 
detailed learning objectives and milestones within each 
competency. We also categorized each learning objective for 
its presence in the early (at the beginning), middle (during 
the preclinical portion), or late (during clinical experiences) 
portion of the curriculum. 

These learning objectives are now being woven into Oregon 
Health & Science University's new curriculum (Table 2). As 
seen in Table 2, we have designated where the specific learning 
activities will be placed in the maj or portions of the curriculum 
from orientation through preclinical sciences, clinical experi- 
ences, and intersession in the fourth year. 

Conclusion and future directions 

Our analysis has shown there are a substantial number of 
informatics competencies and a large body of associated 
knowledge that the 2 1 st century clinician needs to learn and 
apply. From a pedagogical standpoint, there are also issues 
in how to organize, deliver, and assess this content. Cer- 
tainly, one approach is to provide this content as a separate 
course, isolated from the rest of the curriculum. However, 
a better approach would be to tightly and comprehensively 
integrate informatics concepts longitudinally into the learn- 
ing curriculum since clinical informatics is emerging as a 
core competency of medical practice, applicable in all basic 
science disciplines and clinical specialties. 

A next major step for this work will be to develop evalu- 
ation activities for the competencies and learning activities. 



These will vary based on institutional factors (class size, 
whether students are asynchronous, etc), faculty preference, 
and funding limitations (desire to use simulation for many 
things, but cost may be prohibitive). As such, different learn- 
ing activities will require different evaluation methods. For 
example, the competency "Provide care using population 
health management approaches" might be best measured by 
having students describe or select effective system change 
options based on immunizations rates of the clinic to which 
they are assigned, while the competency "Provide clinical 
care via telemedicine and refer those for whom it is neces- 
sary" might best be measured with an objective structured 
clinical examination simulation. 

The presence of these competencies also indicates a 
need for educators who are specialists in informatics to 
(collaboratively with clinical educators) design the learning 
and deliver learning experiences that are appropriate for 
lecture, group discussions, self-paced and self-directed meth- 
ods, and other settings. But informatics is one of those topics 
that is best infused throughout the curriculum, especially in 
clinical settings where it is being used. 

There are a number of future steps for this work. First, 
we recognize that this set of competencies represents the 
views of one medical school's informatics faculty and must 
be validated with informatics experts from other institutions. 
Second, we must evaluate our own implementation of 
this curriculum to determine how these competencies are 
delivered to a medical student audience. Finally, we must 
evaluate this entire process with students from our institution 
and others to determine which competencies and learning 
experiences are most valuable for them in their future clini- 
cal practice. 
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